Distributed storage network and method for communicating data across a plurality of parallel wireless data streams

ABSTRACT

A distributed storage network receives data that is to be transmitted. The data is processed via one or more error-coding dispersed storage functions and sliced into data slices. A certain number of the data slices are selected for wireless communication via a plurality of wireless modules wherein a first wireless module sends a first portion of the data slices, and a second wireless module sends a second portion of the data slices, and so on until the data slices are fully communicated. The wireless modules may be of different hardware, software, protocols, throughputs, bandwidth, speeds, encoding schemes, algorithms, etc. The transmission over several different wireless modules that potentially have different (and potentially changing over time) characteristics may increase both performance and security.

CROSS REFERENCE TO RELATED PATENTS

The present U.S. Utility patent application claims priority pursuant to35 U.S.C. §119(e) to U.S. Provisional Application Ser. No. 61/256,411,entitled “DISTRIBUTED STORAGE NETWORK DATA PROCESSING”, filed Oct. 30,2009, which is hereby incorporated herein by reference in its entiretyand made part of the present U.S. Utility patent application for allpurposes.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

NOT APPLICABLE

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC

NOT APPLICABLE

BACKGROUND OF THE INVENTION

Technical Field of the Invention

This invention relates generally to computing and more particularly tostorage of information.

Description of Related Art

Computing systems are known to communicate, process, and store data.Such computing systems range from wireless smart phones to data centersthat support millions of web searches, stock trades, or on-linepurchases every day. Computing processing is known to manipulate datafrom one form into another. For instance, raw picture data from an imagesensor may be compressed, or manipulated, in accordance with a picturecompression standard to produce a standardized compressed picture thatcan be saved or shared with others. Computer processing capabilitycontinues to advance as processing speed advances and softwareapplications that perform the manipulation become more sophisticated.

With recent advances in computing processing speed and communicationspeed, computers may manipulate real time media from voice to streaminghigh definition (HD) video. Purpose-built communications devices, likethe cell phone, are being replaced or augmented by more general-purposeinformation appliances. For example, smart phones can support telephonycommunications but they are also capable of text messaging, andaccessing the internet to perform functions including email, webbrowsing, remote applications access, and media communications. Mediacommunications may include telephony voice, image transfer, music files,video files, real time video streaming, and more.

Each type of computing system is constructed, and hence operates, inaccordance with one or more communication, processing, and storagestandards. With such standards, and with advances in technology, moreand more of the global information content is being successful convertedinto electronic formats and consumed by users in these electronicformats. For example, more digital cameras are now being sold than filmcameras, thus producing more digital pictures that are shared and viewedelectronically. High growth rates have consistently been observed forweb-based programming. Web-based programming is electronicallydistributing among billions of users a large amount of content over theInternet and this content was, until recently, all broadcast by just afew entities over the air television stations and cable televisionproviders. Digital content standards, such as used in pictures, papers,books, video entertainment, home video, all enable this globaltransformation to a digital format. Electronic content pervasiveness isproducing increasing demands on the storage function of computingsystems.

A typical computer storage function includes one or more memory devicesthat match the needs of the various operational aspects of theprocessing and communication functions. For example, a memory device mayinclude solid-state NAND flash, random access memory (RAM), read onlymemory (ROM), a mechanical hard disk drive, or other types of storage.Each type of memory device has a particular performance range, use case,operational environment, and normalized cost. The computing systemarchitecture optimizes the use of one or more types of memory devices toachieve the desired functional, cost, reliability, performance goals,etc., of the computing system. Generally, the immediacy of accessdictates what type of memory device is used. For example, RAM memory canbe accessed in any random order, all with a constant response time. Bycontrast, memory device technologies that require physical movement suchas magnetic discs, tapes, and optical discs, have a variable responsetimes as the physical movement can take longer than the data transfer,but often these devices can store larger volumes of data in a reliablemanner, long-term manner.

Each type of computer storage system is constructed, and hence operates,in accordance with one or more storage standards. For instance, computerstorage systems may operate in accordance with one or more standardsincluding, but not limited to network file system (NFS), flash filesystem (FFS), disk file system (DFS), small computer system interface(SCSI), internet small computer system interface (iSCSI), file transferprotocol (FTP), and web-based distributed authoring and versioning(WebDAV). An operating systems (OS) and storage standard may specify thedata storage format and interface between the processing subsystem andthe memory devices. The interface may specify a structure, such asdirectories and files. Typically, a memory controller provides aninterface function between the processing function and memory devices.As new storage systems are developed, the memory controller functionalrequirements may change to adapt to new standards.

Memory devices are subject to failure and will eventually fail,especially those memory devices that utilize technologies that requirephysical movement, like a disc drive. For example, it is not uncommonfor a disc drive to suffer from bit level corruption on a regular basis,or suffer from a complete drive failure after an average of three yearsof use. One common solution is to utilize more costly disc drives thathave higher quality internal components. Another solution is to utilizemultiple levels of redundant disc drives to abate these issues byreplicating the data into two or more copies. One such redundant driveapproach is called redundant array of independent discs (RAID). Multiplephysical discs comprise an array where parity data is added to theoriginal data before storing the data across the array. The parity iscalculated such that the failure of one or more discs will not result inthe loss of the original data. The original data can be reconstructedfrom the other working discs if one or more discs fails. RAID 5 usesthree or more discs to protect data from the failure of any one disc.The parity and redundancy overhead reduces the capacity of what threeindependent discs can store by one third (n−1=3−2=2 discs of capacityusing 3 discs). RAID 6 can recover from a loss of two discs and requiresa minimum of four discs with an efficiency of n−2. Typical RAID systemsutilize a RAID control to encode and decode the data across the array.

The drawbacks of the RAID approach include effectiveness, efficiency,and security. As more discs are added, the probability of one or twodiscs failing rises and is not negligible, especially if themore-desirable and less-costly discs are used. When one disc fails, itshould be immediately replaced and the data reconstructed before asecond drive fails, whereby data full recover is no longer an option. Toprovide high reliability over a long time period, it is also common tominor RAID arrays at different physical locations, especially if theRAID array is part of a national level computing system with occasionalsite outages. Unauthorized file access becomes a more acute problem whenwhole copies of the same file are replicated in manylocations/geographies, either on just one storage system site or at twoor more sites. In light of the effectiveness, the efficiency ofdedicating 1 to 2 discs per array for the RAID data-recovery overhead isan issue.

Therefore, a need exists to provide a data storage solution thatprovides more effective timeless continuity of data, minimizes adverseaffects of multiple memory elements failures, provides improvedsecurity, can be adapted to a wide variety of storage system standardsand is compatible with current and anticipated computing andcommunications systems.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

FIG. 1 is a schematic block diagram of an embodiment of a computingsystem in accordance with the invention;

FIG. 2 is a schematic block diagram of an embodiment of a computing corein accordance with the invention;

FIG. 3 is a schematic block diagram of an embodiment of a distributedstorage processing unit in accordance with the invention;

FIG. 4 is a schematic block diagram of an embodiment of a grid module inaccordance with the invention;

FIG. 5 is a diagram of an example embodiment of error coded data slicecreation in accordance with the invention;

FIG. 6 is a flowchart illustrating the segmentation of data in thesystem(s) taught herein;

FIG. 7 is a flowchart illustrating the encryption of data in thesystem(s) taught herein;

FIG. 8 is a flowchart illustrating the decryption of data in thesystem(s) taught herein;

FIG. 9 is a flowchart illustrating the storing of an encryption key inthe system(s) taught herein;

FIG. 10 is a flowchart illustrating the retrieval of an encryption keyin the system(s) taught herein;

FIG. 11 is a schematic block diagram of another embodiment of acomputing system in accordance with the invention; and

FIG. 12 is a schematic block diagram of another embodiment of acomputing system in accordance with the invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a schematic block diagram of a computing system 10 thatincludes one or more of a first type of user device(s) 12, one or moreof a second type of user device(s) 14, at least one distributed storage(DS) processing unit 16, at least one DS managing unit 18, at least onestorage integrity processing unit 20, and a distributed storage network(DSN) memory 22 coupled via a network 24. The network 24 may include oneor more wireless and/or wire-lined communication systems, including oneor more private intranet systems and/or public internet systems; and/orone or more local area networks (LAN) and/or wide area networks (WAN).

The DSN memory 22 includes a plurality of distributed storage (DS) units36 for storing data for the system. Each of the DS units 36 includes aprocessing module and memory and may be located at a geographicallydifferent site than the other DS units (e.g., one in Chicago, one inMilwaukee, one in Tokyo, one in Paris, etc.). The processing module maybe a single processing device or a plurality of processing devices. Sucha processing device may be a microprocessor, micro-controller, graphicsprocessing unit, digital signal processor, microcomputer, centralprocessing unit, field programmable gate array, programmable logicdevice, state machine, logic circuitry, analog circuitry, digitalcircuitry, and/or any device that manipulates signals (analog and/ordigital) based on hard coding of the circuitry and/or operationalinstructions. The processing module may have an associated memory and/ormemory element, which may be a single memory device, a plurality ofmemory devices, and/or embedded circuitry of the processing module. Sucha memory device may be a read-only memory, random access memory,volatile memory, non-volatile memory, static memory, dynamic memory,flash memory, cache memory, and/or any device that stores digital orcomputer information. Note that if the processing module includes morethan one processing device, the processing devices may be centrallylocated (e.g., directly coupled together via a wired and/or wireless busstructure) or may be located in a distributed fashion (e.g., cloudcomputing via indirect coupling via a local area network and/or a widearea network, peer-to-peer, etc.). Further note that when the processingmodule implements one or more of its functions via a state machine,analog circuitry, digital circuitry, and/or logic circuitry, the memoryand/or memory element(s) storing the corresponding operationalinstructions may be embedded within, or external to, the circuitrycomprising the state machine, analog circuitry, digital circuitry,and/or logic circuitry. Still further note that the memory elementstores, and the processing module executes, hard-coded and/oroperational instructions corresponding to at least some of the stepsand/or functions illustrated in FIGS. 1-12.

Each of the user devices 12-14, the DS processing unit 16, the DSmanaging unit 18, and the storage integrity processing unit 20 may be aportable computing device (e.g., one or more of a social networkingdevice, a gaming device, a cell phone, a tablet, a netbook, a smartphone, a personal digital assistant, a digital music player, a digitalvideo player, a laptop computer, a handheld computer, a video gamecontroller, and/or any other portable device that includes a computingcore) and/or a fixed computing device (e.g., one or more of a personalcomputer, a workstation, a computer server, a cable set-top box, asatellite receiver, a television set, a printer, a fax machine, homeentertainment equipment, automotive entertainment device, industrialcontrols, a video game console, and/or any type of home or officecomputing equipment). Such a portable or fixed computing device includesa computing core 26 and one or more interfaces 30, 32, and/or 33. Anembodiment of the computing core 26 will be described with reference toFIG. 2.

With respect to the interfaces, each of the interfaces 30-33 includessoftware and/or hardware to support one or more communication links viathe network 24 and/or directly. For example, interface 30 supports acommunication link (wired, wireless, direct, via a LAN, via the network24, etc.) between the first type of user device 14 and the DS processingunit 16. As another example, DSN interface 32 supports a plurality ofcommunication links via the network 24 between the DSN memory 22 and theDS processing unit 16, the first type of user device 12, and/or thestorage integrity processing unit 20. As yet another example, interface33 supports a communication link between the DS managing unit 18 and anyone of the other devices and/or units 12, 14, 16, 20, and/or 22 via thenetwork 24.

In general and with respect to data storage, the system 10 supportsthree primary functions: distributed network data storage management,distributed data storage and retrieval, and data storage integrityverification. In accordance with these three primary functions, data canbe stored in a distributed manner in a plurality of physically differentlocations and subsequently retrieved in a reliable and secure mannerregardless of failures of individual storage devices, failures ofnetwork equipment, the duration of storage, the amount of data beingstored, attempts at hacking the data, etc.

The DS managing unit 18 performs distributed network data storagemanagement functions, which include establishing distributed datastorage parameters, performing network operations, performing networkadministration, and/or performing network maintenance. The DS managingunit 18 establishes the distributed data storage parameters (e.g.,allocation of virtual DSN memory space, distributed storage parameters,security parameters, billing information, user profile information,etc.) for one or more of the user devices 12-14 (e.g., established forindividual devices, established for a user group of devices, establishedfor public access by the user devices, etc.). For example, the DSmanaging unit 18 coordinates the creation of a vault (e.g., a virtualmemory block) within the DSN memory 22 for a user device (for a group ofdevices, or for public access). The DS managing unit 18 also determinesthe distributed data storage parameters for the vault. In particular,the DS managing unit 18 determines a number of slices (e.g., the numberthat a data segment of a data file and/or data block is partitioned intofor distributed storage) and a read threshold value (e.g., the minimumnumber of valid slices required to reconstruct the full andoriginally-stored data segment).

As another example, the DS managing unit 18 creates and stores, locallyor within the DSN memory 22, user profile information. The user profileinformation includes one or more of authentication information,permissions, and/or the security parameters. The security parameters mayinclude one or more of encryption/decryption scheme, one or moreencryption keys, key generation scheme, and data encoding/decodingscheme.

As yet another example, the DS managing unit 18 creates billinginformation for a particular user, user group, vault access, publicvault access, etc. For instance, the DS managing unit 18 tracks thenumber of times a user accesses a private vault and/or public vaults,which can be used to generate a per-access bill. In another instance,the DS managing unit 18 tracks the amount of data stored and/orretrieved by a user device and/or a user group, which can be used togenerate a per-data-amount bill.

The DS managing unit 18 also performs network operations, networkadministration, and/or network maintenance. As at least part ofperforming the network operations and/or administration, the DS managingunit 18 monitors performance of the devices and/or units of the system10 for potential failures, determines the devices' and/or units+activation status, determines the devices' and/or units' loading, andany other system level operation that affects the performance level ofthe system 10. For example, the DS managing unit 18 receives andaggregates network management alarms, alerts, errors, statusinformation, performance information, and messages from the devices12-14 and/or the units 16, 20, 22. For example, the DS managing unit 18receives a simple network management protocol (SNMP) message regardingthe status of the DS processing unit 16.

The DS managing unit 18 performs the network maintenance by identifyingequipment within the system 10 that needs replacing, upgrading,repairing, and/or expanding. For example, the DS managing unit 18determines that the DSN memory 22 needs more DS units 36 or that one ormore of the DS units 36 needs updating, software upgrades, more memory,etc.

The second primary function (i.e., distributed data storage andretrieval) begins and ends with a user device 12-14. For instance, if asecond type of user device 14 has a data file 38 and/or data block 40 tostore in the DSN memory 22, it sends the data file 38 and/or data block40 to the DS processing unit 16 via its interface 30. As will bedescribed in greater detail with reference to FIG. 2, the interface 30functions to mimic a conventional operating system (OS) file systeminterface (e.g., network file system (NFS), flash file system (FFS),disk file system (DFS), file transfer protocol (FTP), web-baseddistributed authoring and versioning (WebDAV), etc.) and/or a blockmemory interface (e.g., small computer system interface (SCSI), internetsmall computer system interface (iSCSI), etc.). In addition, theinterface 30 may attach a user identification code (ID) to the data file38 and/or data block 40.

The DS processing unit 16 receives the data file 38 and/or data block 40via its interface 30 and performs a distributed storage (DS) process 34thereon (e.g., an error-coding dispersal storage function). The DSprocessing 34 begins by partitioning the data file 38 and/or data block40 into one or more data segments, which is represented as Y datasegments. For example, the DS processing 34 may partition the data file38 and/or data block 40 into a fixed byte size segment (e.g., 2¹ to2^(n) bytes, where n=>2) or a variable byte size (e.g., change byte sizefrom segment to segment, or from groups of segments to groups ofsegments, etc.).

For each of the Y data segments, the DS processing 34 error encodes(e.g., forward error correction (FEC), information dispersal algorithm,or error correction coding) and slices (or slices then error encodes)the data segment into a plurality of error coded (EC) data slices 42-48,which is represented as X slices per data segment in the Y datasegments. The number of slices (X) per segment, which corresponds to anumber of pillars n, is set in accordance with the distributed datastorage parameters and the error coding scheme. For example, if aReed-Solomon (or other FEC scheme) is used in an n/k system, then a datasegment is divided into n slices, where k number of slices is needed toreconstruct the original data (i.e., k is the threshold). As a fewspecific examples, the n/k factor may be 5/3; 6/4; 8/6; 8/5; 16/10.

For each EC slice 42-48, the DS processing unit 16 creates a uniqueslice name and appends it to the corresponding EC slice 42-48. The slicename includes universal DSN memory addressing routing information (e.g.,virtual memory addresses in the DSN memory 22) and user-specificinformation (e.g., user ID, file name, data block identifier, etc.).

The DS processing unit 16 transmits the plurality of EC slices 42-48 toa plurality of DS units 36 of the DSN memory 22 via the DSN interface 32and the network 24. The DSN interface 32 formats each of the slices fortransmission via the network 24. For example, the DSN interface 32 mayutilize an internet protocol (e.g., TCP/IP, etc.) to packetize the ECslices 42-48 for transmission via the network 24.

The number of DS units 36 receiving the EC slices 42-48 is dependent onthe distributed data storage parameters established by the DS managingunit 18. For example, the DS managing unit 18 may indicate that eachslice is to be stored in a different DS unit 36. As another example, theDS managing unit 18 may indicate that like slice numbers of differentdata segments are to be stored in the same DS unit 36. For example, thefirst slice of each of the data segments is to be stored in a first DSunit 36, the second slice of each of the data segments is to be storedin a second DS unit 36, etc. In this manner, the data is encoded andstored in a distributed manner at physically diverse locations toimprove data storage integrity and security. Further examples ofencoding the data segments will be provided with reference to one ormore of FIGS. 2-12.

Each DS unit 36 that receives an EC slice 42-48 for storage translatesthe virtual DSN memory address of the slice into a local physicaladdress for storage. Accordingly, each DS unit 36 maintains a virtual tophysical memory mapping to assist in the storage and retrieval of data.

The first type of user device 12 performs a similar function to storedata in the DSN memory 22 with the exception that it includes the DSprocessing. As such, the device 12 encodes and slices the data fileand/or data block it has to store. The device then transmits the slices35 to the DSN memory via its DSN interface 32 and the network 24.

For a second type of user device 14 to retrieve a data file or datablock from memory, it issues a read command via its interface 30 to theDS processing unit 16. The DS processing unit 16 performs the DSprocessing 34 to identify the DS units 36 storing the slices of the datafile and/or data block based on the read command. The DS processing unit16 may also communicate with the DS managing unit 18 to verify that theuser device 14 is authorized to access the requested data.

Assuming that the user device 14 is authorized to access the requesteddata, the DS processing unit 16 issues slice read commands to at least athreshold number of the DS units 36 storing the requested data (e.g., toat least 10 DS units for a 16/10 error coding scheme). Each of the DSunits 36 receiving the slice read command, verifies the command,accesses its virtual to physical memory mapping, retrieves the requestedslice, or slices, and transmits it to the DS processing unit 16.

Once the DS processing unit 16 has received a read threshold number ofslices for a data segment, it performs an error decoding function andde-slicing to reconstruct the data segment. When Y number of datasegments has been reconstructed, the DS processing unit 16 provides thedata file 38 and/or data block 40 to the user device 14. Note that thefirst type of user device 12 performs a similar process to retrieve adata file and/or data block.

The storage integrity processing unit 20 performs the third primaryfunction of data storage integrity verification. In general, the storageintegrity processing unit 20 periodically retrieves slices 45, and/orslice names, of a data file or data block of a user device to verifythat one or more slices have not been corrupted or lost (e.g., the DSunit failed). The retrieval process mimics the read process previouslydescribed.

If the storage integrity processing unit 20 determines that one or moreslices is corrupted or lost, it rebuilds the corrupted or lost slice(s)in accordance with the error coding scheme. The storage integrityprocessing unit 20 stores the rebuilt slice, or slices, in theappropriate DS unit(s) 36 in a manner that mimics the write processpreviously described.

FIG. 2 is a schematic block diagram of an embodiment of a computing core26 that includes a processing module 50, a memory controller 52, mainmemory 54, a video graphics processing unit 55, an input/output (IO)controller 56, a peripheral component interconnect (PCI) interface (oranother type of interface) 58, an IO interface 60, at least one IOdevice interface module 62, a read only memory (ROM) basic input outputsystem (BIOS) 64, and one or more memory interface modules. The memoryinterface module(s) includes one or more of a universal serial bus (USB)interface module 66, a host bus adapter (HBA) interface module 68, anetwork interface module 70, a flash interface module 72, a hard driveinterface module 74, and a DSN interface module 76. Note the DSNinterface module 76 and/or the network interface module 70 may functionas the interface 30 of the user device 14 of FIG. 1. Further note thatthe IO device interface module 62 and/or the memory interface modulesmay be collectively or individually referred to as IO ports.

The processing module 50 may be a single processing device or aplurality of processing devices. Such a processing device may be amicroprocessor, micro-controller, digital signal processor,microcomputer, central processing unit, field programmable gate array,programmable logic device, state machine, logic circuitry, analogcircuitry, digital circuitry, and/or any device that manipulates signals(analog and/or digital) based on hard coding of the circuitry and/oroperational instructions. The processing module 50 may have anassociated memory and/or memory element, which may be a single memorydevice, a plurality of memory devices, and/or embedded circuitry of theprocessing module 50. Such a memory device may be a read-only memory,random access memory, volatile memory, non-volatile memory, staticmemory, dynamic memory, flash memory, cache memory, and/or any devicethat stores digital information. Note that if the processing module 50includes more than one processing device, the processing devices may becentrally located (e.g., directly coupled together via a wired and/orwireless bus structure) or may be located in a distributed manner (e.g.,cloud computing via indirect coupling via a local area network and/or awide area network). Further note that when the processing module 50implements one or more of its functions via a state machine, analogcircuitry, digital circuitry, and/or logic circuitry, the memory and/ormemory element storing the corresponding operational instructions may beembedded within, or external to, the circuitry comprising the statemachine, analog circuitry, digital circuitry, and/or logic circuitry.Still further note that, the memory element stores, and the processingmodule 50 executes, hard coded and/or operational instructionscorresponding to at least some of the steps and/or functions illustratedin FIGS. 1-12.

FIG. 3 is a schematic block diagram of an embodiment of a dispersedstorage (DS) processing module 34 of user device 12 and/or of the DSprocessing unit 16. The DS processing module 34 includes a gatewaymodule 78, an access module 80, a grid module 82, and a storage module84. The DS processing module 34 may also include an interface 30 and theDSnet interface 32 and/or the interfaces may be part of user device 12or of the DS processing unit 16. The DS processing module 34 may furtherinclude a bypass/feedback path between the storage module 84 and thegateway module 78.

In an example of storing data in one embodiment, the gateway module 78receives an incoming data object (e.g., a data file, a data block, an ECdata slice, etc.) that includes a user ID field 86, an object name field88, and the data field/object 40. The gateway module 78 authenticatesthe user associated with the data object by verifying the user ID 86with the managing unit 18 and/or another authenticating unit. When theuser is authenticated, the gateway module 78 obtains user informationfrom the management unit 18, the user device, and/or the otherauthenticating unit. The user information includes a vault identifier,operational parameters, and user attributes (e.g., user data, billinginformation, etc.) as shown in FIG. 3. A vault identifier identifies avault, which is a virtual memory space that maps to a set of DS storageunits 36. For example, vault 1 (i.e., user 1's DSN memory space)includes eight DS storage units (X=8 wide) and vault 2 (i.e., user 2'sDSN memory space) includes sixteen DS storage units (X=16 wide). Theoperational parameters may include an error coding algorithm, the widthn (number of pillars X or slices per segment for this vault), a readthreshold T, an encryption algorithm, a slicing parameter, a compressionalgorithm, an integrity check method, caching settings, parallelismsettings, and/or other parameters that may be used to access the DSNmemory layer.

The gateway module 78 uses the user information to assign a source nameto the data. For instance, the gateway module 78 determines the sourcename of the data object 40 based on the vault identifier and the dataobject. For example, the source name may contain a data name (blocknumber or a file number), the vault generation (gen) number, thereserved field (resv), an optional file ID, and the vault identifier(ID). The data name may be randomly assigned but is associated with theuser data object 40.

The access module 80 receives the data object 40 and creates a series ofdata segments 1 through Y 90-92 therefrom. The number of segments Y maybe chosen or randomly assigned based on a selected segment size and thesize of the data object. For example, if the number of segments ischosen to be a fixed number, then the size of the segments varies as afunction of the size of the data object. For instance, if the dataobject is an image file of 4,194,304 eight bit bytes (e.g., 33,554,432bits) and the number of segments Y=131,072, then each segment is 256bits or 32 bytes. As another example, if segment size is fixed, then thenumber of segments Y varies based on the size of data object. Forinstance, if the data object is an image file of 4,194,304 bytes and thefixed size of each segment is 4,096 bytes, then the number of segmentsY=1,024. Note that each segment is associated with the source name.

The grid module 82 may pre-manipulate (e.g., compression, encryption,cyclic redundancy check (CRC), etc.) each of the data segments beforeperforming an error coding function of the error coding dispersalstorage function to produce a pre-manipulated data segment. The gridmodule 82 then error encodes (e.g., Reed-Solomon, Convolution encoding,Trellis encoding, etc.) the data segment or pre-manipulated data segmentinto X error coded data slices 42-44. The grid module 64 determines aunique slice name for each error coded data slice and attaches it to thedata slice.

The value X, or the number of pillars (e.g., X=16), is chosen as aparameter of the error coding dispersal storage function. Otherparameters of the error coding dispersal function include a readthreshold T, a write threshold W, etc. The read threshold (e.g., T=10,when X=16) corresponds to the minimum number of error-free error codeddata slices required to reconstruct the data segment. In other words,the DS processing module 34 can compensate for X-T (e.g., 16−10=6)missing error coded data slices per data segment. The write threshold Wcorresponds to a minimum number of DS storage units that acknowledgeproper storage of their respective data slices before the DS processingmodule indicates proper storage of the encoded data segment. Note thatin some embodiments the write threshold is greater than or equal to theread threshold for a given number of pillars (X).

The grid module 82 also determines which of the DS storage units 36 willstore the EC data slices based on a dispersed storage memory mappingassociated with the user's vault and/or DS storage unit 36 attributes.The DS storage unit attributes includes availability, self-selection,performance history, link speed, link latency, ownership, available DSNmemory, domain, cost, a prioritization scheme, a centralized selectionmessage from another source, a lookup table, data ownership, and/or anyother factor to optimize the operation of the computing system. Notethat the number of DS storage units 36 is equal to or greater than thenumber of pillars (e.g., X) so that no more than one error coded dataslice of the same data segment is stored on the same DS storage unit 36.Further note that EC data slices of the same pillar number but ofdifferent segments (e.g., EC data slice 1 of data segment 1 and EC dataslice 1 of data segment 2) may be stored on the same or different DSstorage units 36.

The storage module 84 performs an integrity check on the EC data slicesand, when successful, transmits the EC data slices 1 through X of eachsegment 1 through Y to the DS Storage units. Each of the DS storageunits 36 stores its EC data slice and keeps a table to convert thevirtual DSN address of the EC data slice into physical storageaddresses.

In an example of a read operation, the user device 12 and/or 14 sends aread request to the DS processing unit 16, which authenticates therequest. When the request is authentic, the DS processing unit 16 sendsa read message to each of the DS storage units 36 storing slices of thedata object being read. The slices are received via the DSnet interface32 and processed by the storage module 84, which performs a parity checkand provides the slices to the grid module 82 when the parity check wassuccessful. The grid module 82 decodes the slices in accordance with theerror coding dispersal storage function to reconstruct the data segment.The access module 80 reconstructs the data object from the data segmentsand the gateway module 78 formats the data object for transmission tothe user device.

FIG. 4 is a schematic block diagram of an embodiment of a grid module 82that includes a control unit 73, a pre-data manipulator 75, an encoder77, a slicer 79, a post-data manipulator 81, a pre-data de-manipulator83, a decoder 85, a de-slicer 87, and/or a post-data de-manipulator 89.Note that the control unit 73 may be partially or completely external tothe grid module 82. For example, the control unit 73 may be part of thecomputing core at a remote location, part of a user device, part of theDS managing unit 18, or distributed amongst one or more DS storageunits.

In an example of write operation, the pre-data manipulator 75 receives adata segment 90-92 and a write instruction from an authorized userdevice. The pre-data manipulator 75 determines if pre-manipulation ofthe data segment 90-92 is required and, if so, what type. The pre-datamanipulator 75 may make the determination independently or based oninstructions from the control unit 73, where the determination is baseda computing system-wide predetermination, a table lookup, vaultparameters associated with the user identification, the type of data,security requirements, available DSN memory, performance requirements,and/or other metadata.

Once a positive determination is made, the pre-data manipulator 75manipulates the data segment 90-92 in accordance with the type ofmanipulation. For example, the type of manipulation may be compression(e.g., Lempel-Ziv-Welch, Huffman, Golomb, fractal, wavelet, etc.),signatures (e.g., Digital Signature Algorithm (DSA), Elliptic Curve DSA,Secure Hash Algorithm, etc.), watermarking, tagging, encryption (e.g.,Data Encryption Standard, Advanced Encryption Standard, etc.), addingmetadata (e.g., time/date stamping, user information, file type, etc.),cyclic redundancy check (e.g., CRC32), and/or other data manipulationsto produce the pre-manipulated data segment.

The encoder 77 encodes the pre-manipulated data segment 92 using aforward error correction (FEC) encoder (and/or other type of erasurecoding and/or error coding) to produce an encoded data segment 94. Theencoder 77 determines which forward error correction algorithm to usebased on a predetermination associated with the user's vault, a timebased algorithm, user direction, DS managing unit direction, controlunit direction, as a function of the data type, as a function of thedata segment 92 metadata, and/or any other factor to determine algorithmtype. The forward error correction algorithm may be Golay,Multidimensional parity, Reed-Solomon, Hamming, Bose Ray ChauduriHocquenghem (BCH), Cauchy-Reed-Solomon, or any other FEC encoder. Notethat the encoder 77 may use a different encoding algorithm for each datasegment 92, the same encoding algorithm for the data segments 92 of adata object, or a combination thereof.

The encoded data segment 94 is of greater size than the data segment 92by the overhead rate of the encoding algorithm by a factor of d*(X/T),where d is size of the data segment 92, X is the width or number ofslices, and T is the read threshold. In this regard, the correspondingdecoding process can accommodate at most X-T missing EC data slices andstill recreate the data segment 92. For example, if X=16 and T=10, thenthe data segment 92 will be recoverable as long as 10 or more EC dataslices per segment are not corrupted.

The slicer 79 transforms the encoded data segment 94 into EC data slicesin accordance with the slicing parameter from the vault for this userand/or data segment 92. For example, if the slicing parameter is X=16,then the slicer slices each encoded data segment 94 into 16 encodedslices.

The post-data manipulator 81 performs, if enabled, post-manipulation onthe encoded slices to produce the EC data slices. If enabled, thepost-data manipulator 81 determines the type of post-manipulation, whichmay be based on a computing system-wide predetermination, parameters inthe vault for this user, a table lookup, the user identification, thetype of data, security requirements, available DSN memory, performancerequirements, control unit directed, and/or other metadata. Note thatthe type of post-data manipulation may include slice level compression,signatures, encryption, CRC, addressing, watermarking, tagging, addingmetadata, and/or other manipulation to improve the effectiveness of thecomputing system.

In an example of a read operation, the post-data de-manipulator 89receives at least a read threshold number of EC data slices and performsthe inverse function of the post-data manipulator 81 to produce aplurality of encoded slices. The de-slicer 87 de-slices the encodedslices to produce an encoded data segment 94. The decoder 85 performsthe inverse function of the encoder 77 to recapture the data segment90-92. The pre-data de-manipulator 83 performs the inverse function ofthe pre-data manipulator 75 to recapture the data segment.

FIG. 5 is a diagram of an example of slicing an encoded data segment 94by the slicer 79. In this example, the encoded data segment includesthirty-two bits, but may include more or less bits. The slicer 79disperses the bits of the encoded data segment 94 across the EC dataslices in a pattern as shown. As such, each EC data slice does notinclude consecutive bits of the data segment 94 reducing the impact ofconsecutive bit failures on data recovery. For example, if EC data slice2 (which includes bits 1, 5, 9, 13, 17, 25, and 29) is unavailable(e.g., lost, inaccessible, or corrupted), the data segment can bereconstructed from the other EC data slices (e.g., 1, 3 and 4 for a readthreshold of 3 and a width of 4).

FIG. 6 is a flowchart illustrating the segmentation of data where theaccess module 80 of the DS processing system may receive a data object40, determine how to perform segmentation, and segment the data object40 into data segments in accordance with the segmentation determinationmade by the access module 80.

The method begins with the step 100 where the access module 80 of FIG. 3receives a data object 40 (also shown in FIG. 3) and associated sourcename from the gateway module 78 or any other module of the system. Theaccess module 80 determines the size in bytes (or some other quanta) ofthe data object 40 where the determination is based on incomingmetadata, counting the data object bytes when all the data object byteshave been received, and/or some other algorithmic method, via a step102. The size determination can be made dynamically as the data objectis received by the access module 80, or the size determination can bemade after the full receipt of the data object 40 is complete within (orassociated with) access module 60

The access module 80 may determine or associate metadata for the dataobject where the metadata may include one or more of the data objectsize, a data type indicator, a priority indicator, a security indicator,and/or a user ID, via a step 104. This determination may be based on oneor more of received information appended to the data object, a lookup, acommand, a predetermination, data object inspection, and/or a user vaultentry.

The access module 80 determines a segmentation approach where theapproach may include segmenting the data object 40 into equally sizedfixed data segments 90-92 or segmenting the data object into variablesized data segments 90-92 via steps 106, 108, 110, and/or 116. Thedetermination may be based on one or more of the metadata, a systemloading indicator, received information appended to the data object, alookup, a command, a predetermination, data object inspection, and/or auser vault entry. For example, the access module 80 may choose thefixed-segment-size approach when the system loading indicator indicateslight system loading or if the loading history indicates relativelysteady loading. In another example, the access module 80 may choose thevariable approach when the system loading indicator indicates currentlyheavy system loading such that the incremental load (e.g., of storingthe data object) may not adversely affect the system loading. Note thatfixed data segments may be substantially close in size but not identicalin size. For example, if a 102 unit object was determined to be splitinto 4 fixed or equal parts, the parts would likely be of sizes 25, 25,26, and 26 units. This is the case because the whole does not divideinto equal fixed segments. Also, some segments may be appended withheader or other metadata that leads one segment to be slightly largerthan others. Therefore, when using the term “fixed” herein, the size maybe slightly carrying from segment to segment.

The access module 80 determines a fixed segment size when the accessmodule 80 determines the segmentation approach of segmenting the dataobject into equally-sized fixed data segments, via steps 106, 108, and110 in FIG. 6. The determination may be based on one or more of themetadata, a system loading indicator, received information appended tothe data object, a lookup, a command, a predetermination, data objectinspection, and/or a user vault entry. For example, the access module 80may choose a smaller fixed segment size when the system loadingindicator indicates the system is loading is heavier than average andchoosing smaller segment size will create less incremental loading thanlarger segments.

The access module 80 creates a header and appends the header to the dataobject 40 per a step 112. In another embodiment, the access module 80appends the header to two or more (e.g., as many as all) of the datasegments 90-92 per the step 112. The header may include one or more ofthe data object size, the metadata, the fixed data segment size, and/orthe data segmentation approach.

The access module 80 segments the data object 40 in accordance with thedata segmentation approach and the determined data segment sizes in thestep 114 and sends the segments for further processing by the gridmodule 82 of FIG. 3.

The access module 80 determines or selects a variable pattern when theaccess module 80 determines the segmentation approach of segmenting thedata object into variable sized data segments, via steps 106, 108, and116 in FIG. 6. The variable pattern may be static, random,pseudo-random, cyclical, or dynamic (e.g., the pattern may change toanother pattern over time or as a function of instantaneous systemloading). As examples, the variable pattern may start with smallersegment sizes and ramp upwards in size over time. The variable patternmay start with larger segment sizes and ramp downwards in size overtime. The variable pattern may alternate between larger segment sizesand smaller segment sizes over time. The variable pattern may varysinusoidally or via some other function over time or size. The variablepattern determination may be based on one or more of the metadata, asystem loading indicator, received information appended to the dataobject, a lookup, a command, a predetermination, data object inspection,and/or a user vault entry. For example, the access module 80 may choosea smaller fixed segment size to start with and ramp upwards over timewhen the system loading indicator indicates the system is loading isheavier than average and choosing smaller segment size when that choicewill create less incremental loading than larger segments.

The access module 80 creates a header and appends the header to the dataobject in the step 118. In another embodiment, the access module 80appends the header to two or more (e.g., as many as all) data segmentsin the step 118. The header may include one or more of the data objectsize, the metadata, the data segmentation approach, and/or the variablepattern.

The access module segments the data object in accordance with the datasegmentation approach and the determined variable pattern in the step120 and sends the segments for further processing by the grid module 82of FIG. 3.

Note that the same general computing structure taught herein forenabling functions and modules via input and output interface circuitrycoupled to a central processing unit or like one or more processingmodules may be used to enable operation of the access module 80 in wholeor in part. In other forms, these teachings herein can be used to enablethe entire DS processing unit 34, of which the access module may onlyuse a portion of the overall compute and memory capability of the largerunit 34. Often the central processing unit or one or more processingmodules are coupled to one or more forms of memory devices such asstatic random access memory, dynamic random access memory, non-volatilememory, cache memory, hard drives, optical storage, or other memory.

FIG. 7 is a flowchart illustrating the encryption of data where the gridmodule 82 (see FIG. 3) of the DS processing system receives a datasegment 90-92 and encrypts the data segment 90-92 prior to encoding andslicing each data segment to produce EC data slices 42-48 with improvedsecurity. In particular, the grid module 82 may contain a pre-datamanipulator that may encrypt the data segment as taught herein.

The method begins with the step 122 where the grid module 82 receivesone or more data segments 90-92 from the access module 80 or any othermodule within the system. The grid module 82 partitions the data segment90 or 92 into a first portion and second portion where the portions maybe the same or different sizes via a step 124. In another embodiment,the grid module may partition the data segment 90 or 92 into more thantwo portions to obtain N portions where N is a finite integer greaterthan two. These N portions may be of equal sizes (or nearly equal if thesegment does not divide evenly) or different sizes. In yet anotherembodiment, the grid module 82 partitions the data segment 90 or 92 intotwo or more portions with equal or non-equal sizes based on apartitioning determination. The grid module 82 may determine thepartitioning based on a security procedure, a security indicator, dataobject metadata, a system loading indicator, received informationappended to the data object, a lookup, a command, a predetermination,data object inspection, and/or a user vault entry. For example, thesecurity procedure may indicate that the portion sizes will change withevery data segment by 5%. In other words, the first portion may grow by5% and the second portion may shrink by 5% for the next data segment,until the first portion is 100% and the second portion is 0% in whichcase the security procedure may reverse the process. In another example,even numbered data segments may be partitioned into a 75% first portionand a 25% second portion while odd numbered data segments may bepartitioned into a 15% first portion and an 85% second portion.

Basically, any function over time or any other variable may be appliedto the partitioning scheme so long as the sending and receiving end areaware of the scheme so that encryption and decryption may commenceaccurately. The function applied may also appear random. Meaning, thesending and receiving end may each contain signature analyzers that aresynced to each other, whereby the value of the signature analyzerdetermines the size or fractional size of first and second segments. Forexample, if a signature analyzer sequence with a max value of 100 cyclesthrough the following sequence: 74, 12, 32, 89, 54, then the firstsegment may be set to contain 74% of the total data, 12% of the data,32% of the data and so on whereby the second segment contains theremainder of the data.

The grid module calculates a first portion hash value for the firstportion in a step 126. The hash function type may be stored in the uservault taught herein and associated with the data segment, slice, or filebeing processed.

The grid module 82 may determine a first encryption algorithm based onone or more of a user vault entry, a predetermination, a command, and/ora table lookup utilizing the first portion hash as an index. In a step128, the grid module 82 then produces an encrypted second portion byencrypting the second portion utilizing the first encryption algorithmand an encryption key where the encryption key is based in whole or inpart on the hash value of the first portion. For example, the encryptionkey may be equal to the first portion hash. In another example, theencryption key may be a combination of the first portion hash and asecond number (e.g. a stored value from the user vault, a calculatedvalue) or may be the first portion hash placed through furtherprocessing.

The grid module then calculates a hash of the encrypted second portionvia a step 130. The hash function type may be stored in the user vaultassociated with the data segment. The hash operations for the twosegments may be the same or different.

The grid module 82 may determine a second encryption algorithm based onone or more of a user vault entry, a predetermination, a command, and/ora table lookup utilizing the hash of the encrypted second portion as anindex per a step 132. The second encryption algorithm may be the same ordifferent than the first encryption algorithm. The grid module 82 thenproduces an encrypted first portion by encrypting the first portionutilizing the second encryption algorithm and an encryption key wherethe encryption key is based in whole or in part on the hash of theencrypted second portion. For example, the encryption key may be equalto the hash of the encrypted second portion. In another example, theencryption key may be a combination of the hash of the encrypted secondportion and a second number (e.g. a stored value from the user vault, acalculated value) or post-processed in a similar manner to the firstportion hash.

The grid module 82 then combines the encrypted first portion and theencrypted second portion to produce an encrypted data segment in a step134. Note that an improvement of the method includes providing securitywith efficiency where the size of the encrypted data segment is equal tothe size of the encrypted first portion summed with the size of theencrypted second portion (e.g., no extra bits). Note that security isprovided since the decryption method must be known to decrypt theencrypted data segment. The method of decryption is discussed in greaterdetail with reference to FIG. 8 below.

FIG. 8 is a flowchart illustrating the decryption of data where the gridmodule 82 of the DS processing system recreates an encrypted datasegment (e.g. by retrieving EC data slices, de-slicing the slices, anddecoding the slices) and decrypts the encrypted data segment to producea recreated decrypted data segment. In particular, the grid module'spre-data de-manipulator may decrypt the data segment.

The method begins with the step 136 wherein the grid module's pre-datade-manipulator receives an encrypted data segment from the grid moduledecoder or any other module within the distributed network storagesystem. The grid module 82 partitions the encrypted data segment into anencrypted first portion and an encrypted second portion in a step 138where the portions may be the same or different sizes. In anotherembodiment, the grid module partitions the encrypted data segment intomore than two portions. In yet another embodiment, the grid modulepartitions the encrypted data segment into two or more portions withequal or non-equal sizes based on a partitioning determination (e.g.,the same as the encryption portioning determination) that results in Npartitions where N is a finite integer greater than two. The grid modulemay determine the partitioning based on a security procedure, a securityindicator, data object metadata, a system loading indicator, receivedinformation appended to the data object, a lookup, a command, apredetermination, data object inspection, and/or a user vault entry. Forexample, the security procedure may indicate that the portion sizes willchange with every data segment by 5%. In other words, the first portionmay grow by 5% and the second portion may shrink by 5% for the next datasegment, until the first portion is 100% and the second portion is 0% inwhich case the security procedure may reverse the process. In anotherexample, even numbered encrypted data segments may be partitioned into a75% encrypted first portion and a 25% encrypted second portion while oddnumbered data segments may be partitioned into a 15% encrypted firstportion and an 85% encrypted second portion. However, the partitioningperformed by the decryptor is a function of (or is identical to) theencryption partitioning used when encrypting this data segment or dataobject. Also, the encryption and decryption segmentation parameters maybe set by data segment, data object, data file, user, geographiclocation, address space, or some other parameter.

The grid module 82 then calculates a hash of the encrypted secondportion in a step 140. The hash function type may be stored in the uservault associated with the data segment.

The grid module 82 may then determine a second decryption algorithmbased on one or more of a user vault entry, a predetermination, acommand, and/or a table lookup utilizing the hash of the encryptedsecond portion as an index. However, the decryption algorithm must becompatible with the original encryption operation (see FIG. 7). The gridmodule 82 produces a decrypted first portion by decrypting the encryptedfirst portion utilizing the second decryption algorithm and anencryption key where the encryption key is based in whole or in part onthe hash of the encrypted second portion via a step 142. As an example,the encryption key may be equal to the hash of the encrypted secondportion. In another example, the encryption key may be a combination ofthe hash of the encrypted second portion and a second number (e.g. astored value from the user vault, a calculated value). In otherembodiments, the hash value is placed through algorithmic processing ofsome sort to derive the encryption key used herein.

The grid module 82 then calculates a decrypted first portion hash byperforming a hash on the decrypted first portion via a step 144. Thehash function type may be stored in the user vault associated with thedata segment.

The grid module 82 may determine a first decryption algorithm based onone or more of a user vault entry, a predetermination, a command, and/ora table lookup utilizing the decrypted first portion hash as an index.Again, the encryption and decryption operations should be compatible.However, the first decryption algorithm may be the same or differentthan the second decryption algorithm. The grid module 82 produces adecrypted second portion by decrypting the second portion utilizing thefirst encryption algorithm and an encryption key where the encryptionkey is based in part on the decrypted first portion hash, as shown instep 146. In one example, the encryption key may be equal to thedecrypted first portion hash. In another example, the encryption key maybe a combination of the decrypted first portion hash and a second number(e.g. a stored value from the user vault, a calculated value).

The grid module 82 combines the decrypted first portion and thedecrypted second portion to produce a decrypted data segment in a step148 of FIG. 8. Note that an improvement of this method includesproviding security with efficiency where the size of the encrypted datasegment is equal to the size of the encrypted first portion summed withthe size of the encrypted second portion (e.g., no extra bits). Notethat security is provided where the decryption method must be known todecrypt the encrypted data segment.

FIG. 9 is a flowchart illustrating the storing of an encryption keywhere the DS managing unit may encrypt the key prior to storing the key.

Note that keys may be utilized to encrypt and/or decode controlinformation and/or data content. For example, a public key may beutilized to encrypt a message from any source to a target destinationwhile a private key may be utilized just by a key owner to decrypt themessage for the destination when the message is encrypted utilizing thepublic key. The system may utilize such public/private key pairs forsigning integrity to authenticate units, modules, users, devices, andtransactions. In another example, a secret key may be utilized toencrypt and/or decode data content associated with a secret key owner.For example, a secret key may be utilized to encrypt a series of dataobject data segments prior to encoding, slicing, and storing EC dataslices in the DSN memory. The key owner may utilize the same secret keyto subsequently decrypt retrieved, de-sliced, and decoded encrypted datasegments.

Note that the DS managing unit may enforce permissions such thatretrieving and storing keys is controlled based on the user ID, thesystem element ID, and a permissions list lookup. For example, users mayhave permissions to retrieve and store their own private, public, andsecret keys. In another example, users may have permissions to retrievepublic keys. In yet another example, The DS managing unit may havepermissions to retrieve and store all keys.

The method begins with the step where the DS managing unit receives akey to store from any other system element. The DS managing unitdetermines an encryption method to encrypt the key to produce anencrypted key. Note that the key may be stored in the DSN memory as ECdata slices of the encrypted key to provide improved security.

The encryption methods include a public key method and a password method(the methods will be described below). The DS managing unit determinesthe method to encrypt the key based on one or more of user deviceconnectivity type (e.g., iSCI), a user vault setting, a command, anoperational parameter, availability of a public key, and/or availabilityof a password. For example, the DS managing unit may choose the publickey method when the device connectivity type is iSCI (e.g., no passwordwith iSCI).

The DS managing unit retrieves a public key for the user (or unit) whenthe DS managing unit determines the method to encrypt the key to be thepublic key method. The DS managing unit may retrieve the public key fromthe user vault or it may be included with the key to be stored.

The DS managing unit encrypts the key to be stored to produce anencrypted key utilizing the public key and an encryption algorithm basedon the operational parameters (e.g., stored in the user vault). The DSmanaging unit stores the encrypted key in the user vault. The DSmanaging unit may encode and slice the encrypted key and store the ECdata slices in the DSN memory.

The DS managing unit retrieves a password for the user (or unit) whenthe DS managing unit determines the method to encrypt the key to be thepassword method. The DS managing unit may retrieve the password from theuser vault or it may be included with the key to be stored.

The DS managing unit may retrieve a hash algorithm from the user vault.The DS managing unit calculates a hash of the password to produce astorage key utilizing the hash algorithm.

The DS managing unit encrypts the key to be stored to produce anencrypted key utilizing the storage key and an encryption algorithmbased on the operational parameters (e.g., stored in the user vault).The DS managing unit stores the encrypted key in the user vault. The DSmanaging unit may encode and slice the encrypted key and store the ECdata slices in the DSN memory.

FIG. 10 is a flowchart illustrating the retrieval of an encryption keywhere the DS managing unit may retrieve an encrypted key and decrypt theencrypted key to provide a key to a requester.

The method begins with the step where the DS managing unit receives aretrieve key request from a requester (e.g., any system element). Notethat the key may be previously stored in the user vault and or DSNmemory as an encrypted key as was previously discussed.

The DS managing unit determines the DSN memory and/or user vaultlocation for the associated encrypted key based on one or more of a userID, the requester ID, a key use indicator, a lookup, a command, apredetermination, and/or an identifier associated with the key. The DSmanaging unit retrieves the encrypted key based on the locationdetermination.

The DS managing unit determines a decryption method to decrypt theencrypted key to produce a decrypted key. Note that the encrypted keymay be stored in the DSN memory as EC data slices to provide improvedsecurity.

The decryption methods include a public key method and a password method(the methods will be described below). The DS managing unit determinesthe method to decrypt the key based on one or more of user deviceconnectivity type (e.g., iSCI), a user vault setting, a command, anoperational parameter, availability of a public key, and/or availabilityof a password. For example, the DS managing unit may choose the passwordmethod when a password is available.

The DS managing unit retrieves a private key for the user (or unit) whenthe DS managing unit determines the method to decrypt the key to be thepublic key method. The DS managing unit may retrieve the private keyfrom the user vault or it may be included with the key request.

The DS managing unit decrypts the encrypted key to produce the decryptedkey utilizing the private key and an encryption algorithm based on theoperational parameters (e.g., stored in the user vault). The DS managingunit sends the decrypted key to the requester.

The DS managing unit retrieves a password for the user (or unit) whenthe DS managing unit determines the method to decrypt the key to be thepassword method. The DS managing unit may retrieve the password from theuser vault or it may be included with the key request.

The DS managing unit may retrieve a hash algorithm from the user vault.The DS managing unit calculates a hash of the password to produce astorage key utilizing the hash algorithm.

The DS managing unit decrypts the encrypted key to produce the decryptedkey utilizing the storage key and an encryption algorithm based on theoperational parameters (e.g., stored in the user vault). The DS managingunit sends the decrypted key to the requester.

FIG. 11 is a schematic block diagram of another embodiment of acomputing system where a user device 12 may utilize two or moresimultaneous wireless connections thru one or a plurality of modules 192to store and/or retrieve EC data slices to/from the DSN memoryassociated with DS storage unit(s) 36. Such a system may provideimproved performance and security.

The system includes a user device 12, a network 24, and the DSN memorywhich contains one or more DS units 36. The network 24 may include oneor more wireless networks 1 through n to accommodate wirelessconnections between the user device and the DSN memory. While FIG. 11shows n wireless signals and n DS units 36, it should be understood thatin another embodiment one wireless signal can serve a plurality of DSunits 36 or one wireless module may time multiplex or frequencymultiplex process multiple of the wireless signals shown in FIG. 11.Therefore, the value of n across all of the modules 192, wirelesssignals, and DS units 36 need not be equal.

The user device 12 includes the DS processing unit/function 34 (see FIG.1), the DSN interface 32 (see FIG. 1), and one or more wireless modules192 (1-n modules where n is a finite positive integer). In anembodiment, the wireless modules 192 may be implemented as n hardwaretransceivers or fewer than n frequency multiplexed, time multiplexed, orthe like. In another embodiment, the wireless modules 1-n may beimplemented as n software modules operating on one hardware transceiver(e.g., a software defined radio (SDR)). In yet another embodiment, thewireless modules 1-n may be implemented as n software modules operatingon two or more hardware transceivers (e.g., software defined radios).

The wireless module 192 communicates wireless signals with the network24 and may operate in accordance with one or more wireless industrystandards including but not limited to universal mobiletelecommunications system (UMTS), global system for mobilecommunications (GSM), long term evolution (LTE), wideband code divisionmultiplexing (WCDMA), IEEE 802.11, IEEE 802.16, WiMax, bluetooth, or anyother LAN, WAN, PAN or like wireless protocol. Therefore any two, four,or any number of wireless modules in FIG. 12 may be powered by one ormore different wireless protocols.

In one embodiment, wireless module 1 communicates (e.g., transmits andreceives) wireless signals 1 with the network. In this embodiment,wireless module 2 communicates (e.g., transmits and receives) wirelesssignals 2 with the network. In general, wireless modules 1-n communicate(e.g., transmits and receives) wireless signals 1-n with the network inthis embodiment.

Wireless modules 1-n may utilize the same or different wireless industrystandards. For example, wireless module 1 may utilize GSM and wirelessmodule 2 may simultaneously utilize IEEE802.16. Wireless modules 1-n mayutilize similar or different performance levels (e.g., speed in bits persecond) of the wireless signals 1-n. For example, wireless module 1 maycommunicate at 100 kilo bits per second (Kbps) via wireless signals 1 inaccordance with the WCDMA standard and wireless module 2 maysimultaneously communicate at 3.3 mega bits per second (Mbps) viawireless signals 2 in accordance with IEEE 802.11 standard. In anotherexample, wireless module 1 and wireless module 2 may both utilize thesame portion of the network in accordance with the IEEE802.16 standardbut operate at different performance levels. For instance, wirelessmodule 1 may communicate at 350 kilo bits per second via wirelesssignals 1 in accordance with the IEEE 802.16 standard and wirelessmodule 2 may simultaneously communicate at 675 kilo bits per second viawireless signals 2 in accordance with IEEE 802.16 standard. Since SDR'sare possible in some embodiments, such protocols may be changed overtime according to a predetermined security algorithm whereby theprotocol on one or more wireless channels is changing over time.

The DS processing unit/function 34 determines which of the wirelessmodules 1-n to utilize to transfer slices to and from the DSN memory.The DS processing unit/function 34 may determine or select two or moresimultaneous wireless paths. For example, the DS processingunit/function 34 may determine to utilize wireless module 1 tocommunicate slice 1 over wireless signal 1, wireless module 2 tocommunicate slice 2 over wireless signal 2, wireless module 3 tocommunicate slice 3 over wireless signal 3, etc. and wireless module nto communicate slice n over wireless signal n. In another example, theDS processing may determine to utilize wireless module 1 to communicateslice 1 through slice 10 over wireless signal 1 and wireless module 2 tocommunicate slice 11 through slice n over wireless signal 2, etc.Therefore, the various wireless channels may communicate differentquantities of data over different times or bandwidth availability andmay change protocols or encryption techniques in order to improvesecurity.

In an example of operation to illustrate an embodiment method, the DSprocessing unit/function 34 creates n slices for storage in the DS units36 of the DSN memory by creating a data segment, encoding the segment,and slicing the encoded segment into data slices. The DS processingunit/function 34 determines performance requirements (e.g., storage andretrieval latencies) and security requirements (e.g., higher or lowerlevel of security) based on user vault information and/or metadataassociated with the processed data object 40. The DS processingunit/function 34 determines which wireless modules 192 to utilize tocommunicate the n slices to the DSN memory. The determination may bebased on one or more of the performance requirements, the securityrequirements, and performance indicators for each or some of thewireless modules 192, and/or security indicators for each of thewireless modules 192. The DS processing unit/function 34 determines amapping of the n slices to the determined wireless modules 192 where thedetermination may be based on one or more of the performancerequirements, the security requirements, performance indicators for eachof the wireless modules 192, and/or security indicators for each of thewireless modules 192. The DS processing unit/function 34 sends theslices with a store command to the DSN memory via the determinedwireless modules 192 and the determined mapping of the n slices to thedetermined wireless modules 192.

In another example of operation to illustrate yet anotherembodiment/method, the DS processing unit/function 34 retrieves n slicesfrom the various DS unit(s) 36 of the DSN memory in FIG. 11. The DSprocessing unit/function 34 determines performance requirements (e.g.,storage and retrieval latencies) and security requirements (e.g., higheror lower level of security) based on user vault information and/ormetadata associated with the data object, or some other method. The DSprocessing unit/function 34 determines the wireless modules 192 toutilize to retrieve the n slices from the DS unit(s) 36 if the DSNmemory. The determination may be based on one or more of the performancerequirements, the security requirements, and performance indicators foreach of the wireless modules 192, and/or security indicators for each ofthe wireless modules 192. The DS processing unit/function 34 determinesa mapping of the n slices to the determined wireless modules 192 wherethe determination may be based on one or more of the performancerequirements, the security requirements, performance indicators for eachof the wireless modules 192 n, and/or security indicators for each ofthe wireless modules 192. The DS processing unit/function 34 sends 192slice retrieval commands to the DSN memory or individual DS units 26 viathe determined wireless modules 192 and the determined mapping of the nslices to the determined wireless modules.

FIG. 12 is a schematic block diagram of another embodiment of acomputing system where error coded (EC) data slices are created anddistributed to the DS unit(s) 36 within the DSN memory by one or moreuser device(s) 12. The EC data slices are video from a videosurveillance camera, a television event, stadium camera coverage at alive event (e.g., football or boxing match) or some other stream ofvideo information. The system includes user devices 1-D, the network 24,the storage integrity processing unit 20, the DSN memory, and a player.

In an embodiment, user device 1 includes the computing core 26(containing or connected to the DS processing unit/function 34) and theDSN interface 32. An external video camera 194 interfaces with thecomputing core via one of the interfaces discussed with reference toFIG. 2, such as USB, firewire, PCI, wireless connections, or likeinterfaces. In another embodiment, user device D includes an integratedcamera 196, the computing core 22 (along with the associated DSprocessing unit/function 34), and the DSN interface 32. Theinternal/integrated video camera also may interface with the computingcore 22 via one of the interfaces discussed with reference to FIG. 2. Inyet another embodiment, the computing core 22 includes the camera 196(this configuration is not specifically shown in FIG. 12).

Either or both cameras 194-196 may output standard definition (SD)and/or high definition (HD) video utilizing one of a plurality of videocodec or video compression algorithms and may interface with thecomputing core 22 via an analog and/or digital interface that is wiredand/or wireline. The computing core 22 may communicate control andmetadata information with their respective cameras 194-196. The controlinformation may include operational instructions including but notlimited to the video compression algorithm to utilize, a camera positionschedule, pan left, pan right, zoom in, zoom out, change from visiblemode to infrared mode, match a pattern, new software load, etc. Themetadata information may include a timestamp, location information,pattern recognition information, camera setting information, cameradirection, camera type, camera software version, 3D rendering data,depth data, facial recognition flags/alerts or information, internetprotocol address, camera ID, etc. The camera 196 may operate inaccordance with the control information and send video and metadata tothe computing core 22.

The computing core 22 includes the DS processing unit/function 34. TheDS processing unit/function 34 may receive the video and metadata from acorresponding one or more cameras 196. The DS processing unit/function34 determines DSN operational parameters (e.g., number of pillars,encoding method, slicing method, encryption information, and DSNdestinations) that are used for transferring and storing camera video.The determination may be based on one or more of an assignment by the DSmanaging unit, a predetermination, network performance, DSN memoryavailability, and/or information from the player.

The DS processing unit/function 34 creates EC data slices of the videoand metadata based on the DSN operational parameters. The DS processingunit/function 34 sends the EC data slices with a store command via thenetwork to the DSN destinations (e.g., the DSN memory 22, the player 198for live viewing a local caching, other camera-equipped user devices forcaching and processing). The DS processing unit/function 34 may appendthe operational parameters to the EC data slices such that the playercan readily decode the slices and play back the video.

The player 198 includes a DS processing unit/function 34 that isequipped or associated with the computing core 26, the DSN interface 32,and may include an internal or external display(s) 200 to display video.The player DS processing unit/function 34 may receive slices from one ormore cameras 196 or DSN memory 22, de-slice and decode the slices inaccordance with the operational parameters as taught herein, and routethe resulting video to the display(s) 200. The player 198 may furtherprocess the video based in part on the metadata to analyze the video(e.g., look for patterns, detect faces, detect objects, detect events,time stamp certain events, etc.). The player 198 may send controlinformation to the camera via the network to improve or change/programthe operation of the camera 194 and/or 196.

The storage integrity processing unit 20 may determine when and howslices stored in the DSN memory are to be deleted, where thedetermination may be based on one or more of video storage agerequirements (e.g., evidence/records retention policy), a currenttimestamp, a stored video timestamp, the metadata, a command, a commandfrom the player, a command from the camera, a predetermination, and/or aDSN memory availability indicator. For example, the storage integrityprocessing unit 20 may identify slices of video that are greater thanseven years old and the video storage requirements specify seven years.Or, usage data of the video may show that nobody has accessed the videoin a threshold amount of time or at a rate that warrants retention. Inthese events, the storage integrity processing unit 20 sends a deletecommand to the DSN memory for the determined slices to be deleted, or atleast removed from functional memory and sent to backup storage (liketape files or archival disks).

The storage integrity processing unit 20 may determine slicesrepresenting video stored in the DSN memory that are to be retrieved,decoded, recompressed, etc., with different video compression algorithmsor encodes and may store those new files to the DSN memory 22. Thedetermination for this processing may be based on one or more of videostorage age requirements (e.g., evidence/records retention policy), acurrent timestamp, a stored video timestamp, type of usage, frequency ofusage, the metadata, a command, a command from the player, a commandfrom the camera, a predetermination, and/or a DSN memory availabilityindicator. For example, the storage integrity processing unit 20 mayidentify slices of video that are the oldest or least used/accessed andthe DSN memory availability indicator may indicate a shortage of memorywhereby these files need to be compressed, reduced in quality, removed,etc.

In these cases, the storage integrity processing unit 20 sends aretrieve command to the DSN memory 22 for the determined slices to berecompressed. The storage integrity processing unit 20 receives theslices, de-slices and decodes the slices to produce the video inaccordance with the operational parameters. The storage integrityprocessing unit 20 also determines a new video compression algorithmbased on the metadata, a command, a command from the player, a vaultlookup, usage patters, usage frequency, usage quantity, time accessed, acommand from the camera, a predetermination, and/or a DSN memoryavailability indicator. The storage integrity processing unit 20recompresses the video with a new video compression algorithm that willprovide an improvement in memory availability or utilization (e.g.,utilize less DSN memory space or free up more accessible or faster spacefor content that can be processed or delivered faster).

The storage integrity processing unit 20 determines the new DSNoperational parameters to create slices from the recompressed videobased on the metadata, a command, a command from the player, a vaultlookup, a command from the camera, a predetermination, and/or a DSNmemory availability indicator or other parameters taught herein. Thestorage integrity processing unit 20 encodes and slices the recompressedvideo in accordance with the new DSN operational parameters to producenew slices. The storage integrity processing unit sends the new sliceswith a store command to the DSN memory to store the new data slices.Note, the cameras taught in FIG. 12 may be any camera that captures anykind of image in any kind of format or spectrum. So, cameras 196 may beany sensing device, such as video cameras, professional film cameras. 3Dcameras, embedded low cost laptop cameras, security cameras, scientificcameras that capture other spectrums (infrared, gamma ray, ultraviolet,microwave, etc.), night spectrum cameras, heat sensors, simple motiondetectors, thermometers, microphones, or any other camera or combinationof devices that track audio and/or visual data, spectrum data, orchanges in such data over time. The system taught in FIG. 12 allows realtime or near real time information to be processed and sent using thesegment and slice storage and security methodology taught herein. Nearreal time generally means any processing done within a few seconds to afew minutes of the capture of the real time data. However, if the caseof scientific data, as in transmission from satellite or space boundobjects, the time may take longer. Audio/visual information as usedherein means any data or information that contains one or both of audioor visual information.

As may be used herein, the terms “substantially” and “approximately”provides an industry-accepted tolerance for its corresponding termand/or relativity between items. Such an industry-accepted toleranceranges from less than one percent to fifty percent and corresponds to,but is not limited to, component values, integrated circuit processvariations, temperature variations, rise and fall times, and/or thermalnoise. Such relativity between items ranges from a difference of a fewpercent to magnitude differences. As may also be used herein, theterm(s) “coupled to” and/or “coupling” and/or includes direct couplingbetween items and/or indirect coupling between items via an interveningitem (e.g., an item includes, but is not limited to, a component, anelement, a circuit, and/or a module) where, for indirect coupling, theintervening item does not modify the information of a signal but mayadjust its current level, voltage level, and/or power level. As mayfurther be used herein, inferred coupling (i.e., where one element iscoupled to another element by inference) includes direct and indirectcoupling between two items in the same manner as “coupled to”. As mayeven further be used herein, the term “operable to” indicates that anitem includes one or more of power connections, input(s), output(s),etc., to perform one or more its corresponding functions and may furtherinclude inferred coupling to one or more other items. As may stillfurther be used herein, the term “associated with”, includes directand/or indirect coupling of separate items and/or one item beingembedded within another item. As may be used herein, the term “comparesfavorably”, indicates that a comparison between two or more items,signals, etc., provides a desired relationship. For example, when thedesired relationship is that signal 1 has a greater magnitude thansignal 2, a favorable comparison may be achieved when the magnitude ofsignal 1 is greater than that of signal 2 or when the magnitude ofsignal 2 is less than that of signal 1.

The present invention has also been described above with the aid ofmethod steps illustrating the performance of specified functions andrelationships thereof. The boundaries and sequence of these functionalbuilding blocks and method steps have been arbitrarily defined hereinfor convenience of description. Alternate boundaries and sequences canbe defined so long as the specified functions and relationships areappropriately performed. Any such alternate boundaries or sequences arethus within the scope and spirit of the claimed invention. Furthermore,the system taught herein may be referred to either as dispersed storagenetwork or distributed storage networks.

The present invention has been described above with the aid offunctional building blocks illustrating the performance of certainsignificant functions. The boundaries of these functional buildingblocks have been arbitrarily defined for convenience of description.Alternate boundaries could be defined as long as the certain significantfunctions are appropriately performed. Similarly, flow diagram blocksmay also have been arbitrarily defined herein to illustrate certainsignificant functionality. To the extent used, the flow diagram blockboundaries and sequence could have been defined otherwise and stillperform the certain significant functionality. Such alternatedefinitions of both functional building blocks and flow diagram blocksand sequences are thus within the scope and spirit of the claimedinvention. One of average skill in the art will also recognize that thefunctional building blocks, and other illustrative blocks, modules andcomponents herein, can be implemented as illustrated or by discretecomponents, application specific integrated circuits, processorsexecuting appropriate software and the like or any combination thereof.

What is claimed is:
 1. A method for outputting data for storage by auser device in a distributed storage network (DSN), the methodcomprising: processing the data into a plurality of sets of encoded dataslices using an error coding dispersal storage function, wherein a datasegment of the data is encoded into one of the plurality of sets ofencoded data slices; identifying distributed storage (DS) units of theDSN for storing the plurality of sets of encoded data slices, wherein afirst DS unit of the identified DS unit to store a first encoded dataslice of each set of at least some of the plurality of sets of encodeddata slices, a second DS unit of the identified DS unit to store asecond encoded data slice of each set of the at least some of theplurality of sets of encoded data slices; and a third DS unit of theidentified DS unit to store a third encoded data slice of each set of atleast some of the plurality of sets of encoded data slices; determiningperformance requirements regarding storing the plurality of sets ofencoded data slices in the identified DS units; determining securityrequirements of the plurality of sets of encoded data slices based, atleast in part, on vault information associated with the data segment; inaccordance with the performance requirements and the securityrequirements, mapping the at least some of the plurality of sets ofencoded data slices to particular wireless modules of the user devicefor substantially simultaneous wireless transmission of the at leastsome of the plurality of sets of encoded data slices to the identifiedDS units via one or more networks; transmitting, by a first wirelessmodule of the wireless modules, the first encoded data slice of each ofthe at least some of the plurality of sets of encoded data slices to thefirst DS unit; transmitting, by the first wireless module, the secondencoded data slice of each of the at least some of the plurality of setsof encoded data slices to the second DS unit; and transmitting, by asecond wireless module of the wireless modules, the third encoded dataslice of each of the at least some of the plurality of sets of encodeddata slices to the third DS unit.
 2. The method of claim 1, wherein theperformance requirements comprises one or more of: a wirelesscommunication protocol; a desired security level; a desired data rate;an acceptable latency; and a desired bandwidth.
 3. The method of claim 1wherein the identifying the wireless modules further comprises:comparing the performance requirements with performance capabilities ofa plurality of wireless modules; and based on the comparing, selectingthe particular wireless modules from the plurality of wireless modules.4. The method of claim 1 further comprises: transmitting, by the firstwireless module, the first and second encoded data slices in accordancewith a first wireless communication protocol; and transmitting, by thesecond wireless module, the third encoded data slice in accordance witha second wireless communication protocol.
 5. The method of claim 1further comprises: transmitting, by the first wireless module, the firstand second encoded data slices at a first data rate; and transmitting,by the second wireless module, the third encoded data slice at a seconddata rate, wherein the first data rate is greater than the second datarate.
 6. The method of claim 1 further comprises: transmitting, by athird wireless module of the wireless modules, fourth encoded dataslices of the at least some of the plurality of sets of encoded dataslices to a fourth DS unit of the identified DS units; and transmitting,by a fourth wireless module of the wireless modules, fifth encoded dataslice of the at least some of the plurality of sets of encoded dataslices to a fifth DS unit of the identified DS units, for eachassignment of one of the wireless modules to the one or more of theidentified DS units, determining which of the plurality of sets ofencoded data slices to be stored in the one or more identified DS units.7. A user device of a distributed storage network (DSN) comprising: aplurality of wireless modules; an interface for sending data slices; anda distributed storage processing module coupled to the interface andbeing operable to: process the data into a plurality of sets of encodeddata slices using an error coding dispersal storage function, wherein adata segment of the data is encoded into one of the plurality of sets ofencoded data slices; identify distributed storage (DS) units of the DSNfor storing the plurality of sets of encoded data slices, wherein afirst DS unit of the identified DS unit to store a first encoded dataslice of each set of at least some of the plurality of sets of encodeddata slices, a second DS unit of the identified DS unit to store asecond encoded data slice of each set of the at least some of theplurality of sets of encoded data slices; and a third DS unit of theidentified DS unit to store a third encoded data slice of each set of atleast some of the plurality of sets of encoded data slices; determineperformance requirements regarding storing the plurality of sets ofencoded data slices in the identified DS units; determine securityrequirements of the plurality of sets of encoded data slices based, atleast in part, on vault information associated with the data segment inaccordance with the performance requirements and the securityrequirements, map the at least some of the plurality of sets of encodeddata slices to particular wireless modules of the user device forsubstantially simultaneous wireless transmission of the at least some ofthe plurality of sets of encoded data slices to the identified DS unitsvia one or more networks; transmit, by a first wireless module of theplurality of wireless modules, the first encoded data slice of each ofthe at least some of the plurality of sets of encoded data slices to thefirst DS unit; transmit, by the first wireless module, the secondencoded data slice of each of the at least some of the plurality of setsof encoded data slices to the second DS unit; and transmit, by a secondwireless module of the plurality of wireless modules, the third encodeddata slice of each of the at least some of the plurality of sets ofencoded data slices to the third DS unit.
 8. The user device of claim 7further comprises: transmit, by the first wireless module, the first andsecond encoded data slices in accordance with a first wirelesscommunication protocol; and transmit, by the second wireless module, thethird encoded data slice in accordance with a second wirelesscommunication protocol.
 9. The user device of claim 7 further comprises:transmit, by the first wireless module, the first and second encodeddata slices at a first data rate; and transmit, by the second wirelessmodule, the third encoded data slice at a second data rate, wherein thefirst data rate is greater than the second data rate.
 10. The userdevice of claim 7 further comprises: transmit, by a third wirelessmodule of the plurality of wireless modules, fourth encoded data slicesof the at least some of the plurality of sets of encoded data slices toa fourth DS unit of the identified DS units; and transmit, by a fourthwireless module of the plurality of wireless modules, fifth encoded dataslice of the at least some of the plurality of sets of encoded dataslices to a fifth DS unit of the identified DS units.
 11. The userdevice of claim 7, wherein the distributed storage processing module isfurther operable to identify the wireless modules further by: comparingthe performance requirements with performance capabilities of aplurality of wireless modules; and based on the comparing, selecting theidentified wireless modules from the plurality of wireless modules.